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The International Association of Experimenting 
Schools for Adolescents is a "consortium of persons seeking to 
develop experimental schools which will facilitate the <jrowth of 
adolescents capacities to live successfully In modernizing 
societies." Simultaneous with the developmerrt of the As.^ociaticn was 
a concern by its founders that an evaluation coepcnent be designed 
and installed to provide for a "dynamic diagnosis" of the system. 
Represented is a first attempt to think through th^, i^ction of 
evaluation as it might apply to the Association^ tc define the 
context upon which the eT^aluation will be installedr to consider 
problems related to design and instrumentation, and to anticipate 
those processes which will likely be necessary to integrate an 
evaluation plan into the regular operation of the Association and 
each of its member organizations* The purpose of evaluation will be 
briefly discussed , as will its specific uses in experioenting and/or 
alternative schools. Evaluation will be viewed in the context of 
development, with special att^mtion being paid to the objectives and 
process of building a design. (Author) 
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INTRODUCTION 

The International Association of Experimenting Schools for 
Adolescents is a ^'consortium of persons seeking to develop experimental 
schools which will facilitate the growth of adolescents capacities to live 
successfully in modernizing societies/' Conceived -less than three years ago, 
and having undergone continuous development since then on both sides of the 
Atlantic^, the idea is no longer embryonic, but still is clearly in the very 
early stages of its formative existence. 

Simultaneous with the development of the Association was a concern 
by its founders that an evaluation component be designed and installed to 
provide for a "dynamic diagnosis** of the system. 

That which follows respresents a first attempt to think through 
the notion of evaluation as it might apply to the Association, to define the 
context upon which the evaluation will be installed, to consider problems related 
to design and instrumentation, and to anticipate those processes which will likely 
be necessary to integrate an evaluation plan into the regular operation of the 
Association and each of its member organizations. 

The purpose of evaluation will be briefly discussed, as will its 
specific uses in experimenting and/or alternative schools. Evaluation will 
be viewed in the context of development, with special attention being paid to 
the objectives and process of building a design. 
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THE PURPOSE OF AN EVALUATION SYJiTEM 

The primary purpose of an evaluation system is to provide information 
to dec is eon-makers for use in planning, programming, implementing, and re- 
cycling activities. Stuff lebeam has defined evaluation as the collecting, 
organizing, analyzing, and reporting of timely,* valid, and credible in- 
formation. In this context it should be stressed than an evaluation syster 
must be an integral part of the operation of any experimenting school, and 
of an association designed to foster the growth of such schools. 

In the United States, one is likely, to be more familiar with the 
term "alternative schools** than '"experimenting schools.'' Many so-called 
alternative schools are merely existing structures, updated by adding a 
few different course offerings. A true alternative program views the school 
itself as an experimental proposition. Experimenting schools, as defined by 
the Association, must be willing to confront the basic issue: how can 
schools be designed and operated to satisfy the needs of all young people 
to learn to live successfully and to participate fully in a modernizing 
society?" An alternative school which is c^eveloped to deal with this issue 
is the genuine article, and as such might be synonymous with an experimenting 
school . 

One disadvantage of the term "experimenting," however, is its 
close association with the notion of laboratory research. Research creates 
new knowledge or adds to the existing base of knowledge. It need not create 
or add knowledge useful to a deci s ion-rnaker , or to anyone, for that matter. 
Research has a place in the Association and each of its member organizations, 
but it is separate and apart from the need for an evaluation system. 
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One reason we know so little about whether, what, or why alternative 
schools have produced, is that those responsible for directing them have 
been loathe to allow eva^'luation to take place. This over-protect iveness 
is due largely to confusion between research and evaluation, and the 
feeling that what is occurring in an alternative school cannot be 
accurately reflected or understood by anyone other than an actual program 
participant* Some alternative school directors have been stung by in" 
sensitive researchers or evaluators and may be justified in their distrust, 
while others wish to keep their "experiments'V pure from contamination for 
as long as possible. Whateves^ the reason, failure to provide for d system 
of evaluating an alternative school from the outset is a counter-'productive 
move. Hickey, in his position paper on Evaluati n g Alternative Schools has 
listed four reasons why such evaluation is imperative: 

1. Evaluation provides information related to internal 
program improvement, which, in turn, relates to the 
on-going planning process. 

2. Evaluation serves as a basis for establishing the 
credibility of the school, which must meet the 
demands of a variety of "publics/* 

3. A primary rationale for the existence of most 
alternative schools is that they become the means 
of the process by which public education evolves- 

4. Evaluation of student progress is difficult without 

an adequate understanding of where the program, itself 
stands . 

In the original proposal for the creation of the Association, 
Sanders, Smilansky, and Coleman developed what they termed "dynamic 
diagnosis," as "a foundation for a helping system.'* They based their 
notion on the assumption that: 

I. "...the evaluation would be a comprehensive, diversified, 
longitudinal process shared cooperatively by ...students. 



teachers and tutors, administrators, and parents.'* 



'••••all participants in the school will share responsibility 
for establishing criteria, and developincj and practicing 
diagnostic and evaluation designs/* 

"•.•there will be a system of diagnosis that will be 
supportive of persons in perceiving, -eflecting, and 
declsion-^making^ . / * 
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•FOCUS OF THE EVALUATION 

If the purpose of an evaluation system is to provide information 
upon which to base decisions, then the focus of an evaluation must be upon 
the decisions to be made. The Braybrooke and Lindblom model of public 
policy decision settings adapted to the educational milieu by Stuff lebeam 
may serve as a useful vehicle for establishing the relationship between 
the Association and its evaluation system. 

Braybrooke and Lindblom's model (Figure I) consists of an ordinate 
(Y) axis representing the level of understanding held by the decision maker, 
(the lowest point being the origin) and an abscissa (X) axes repre^senting 
the degree of change expected to occur, (again, Increasing in size as it 
moves away from the origin). Four basic decision settings are established: 
Homeostasis (Y high, X small); Incremental ism (Y low, X small); Metamorphism 
(Y high, X large); and Neomobilism (Y low, X high). In the homeostatic 
setting, the activity is restorative and the purpose is maintenance; in 
the Incremental, the activity is developmental and the purpose, continuous 
improvement; in the metamorphic, the activity is Utopian, the purpose, 
complete change; and in the neomobi 1 istic setting, the activity Is 
innovative and the purpose is to invent, test, diffuse, and eventually 
adopt certain findings. 

In its present stage of development, the Association has moved 
into a neomobi 1 istic decision setting. During an earlier period, while the 
document ,^ntitled School as an Experimental Propos i ti on was be ing conceptual i zed 
and written, placement in the incremental setting might have been more appropriate.. 
At that time, the set of propositions upon which the Association was to be 
built was being developed and undergoing continuous refining. Now, however. 
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- FIGURE I - 



Braybrooke and Lindblom^s Model of Decision Settings 
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these propositions have been established, and the formal founding of the 
Association has signalled the beginning of planned change (innovation) ^ 
the purpose of which is to develop (invent) strategies based upon the 
accepted set of propositions, test them> and diffuse the results to each 
of the Association's member organizations. This closely parallels the 
Association's initial proposal that **..,a cooperative effort of social 
scientists and field-oriented partners .be undertaken to study . schools 
as an experimental proposition." 

In The Bases for Decision Making and Planned Change , Hock, Kean, 
and Smith articulated six components of the decision process. These 
components are (1) awareness, (2) intelligence, (3y design, Cl) choice, 
(5) implementation, and (6) reflection. In order to avoid the often 
repeated developer's mistake of re-inventing the wheel, it should be 
noted that the awareness stage (cognizance of the existence of a basic' 
uneasiness about a system) and the intelligence stage (identification of 
-the known parameters of a problem and its context) have been completed on 
the macro (Association) level. These two stages, similar to what 
Stufflebeam calls "Context Evaluation" need to be documented on the 
micro (member organization) level to establish raison d'etre for each 
member's participation, This should be able to be accompl i shed, however, 
by simply collecting local data, and ascertaining that the data and the 
local agency's willingness to deal with the problem(s) it reflects relate 
to the purpose of the Association. It must be assumed that each prospective 
member organization will be ready to embark upon -the design component when 
the Association does. 



BUILDING TH;£ DESIGN 

The approach to the development of an evaluation design for the 
Association is tmmediately blocked by one major obstacle: the design must 
provide information useable both locally and internationally. As such, the 
design must speak to the specific niaeds of decision makers in each participating 
member organization as well as it having to produce information in a form 
which will enable it to be shared with all other member organizations and be 
comparable with their data bases. Critical to this is the development of the 
Association's minimum data requirements and the delineation of overall in- 
formation needs* 

4 

Though, as stated in the previous section, each member organization 
will have already "defined** its system prior to the decision to participate 
in the Association, it may be necessary to go back and restate the data in 
a universally useable form and format. The Association must first decide 
about which key descriptors it will need data. In addition to defining 
these key descriptors, agreement must also be reached upon evaluation 
policies and basic evaluation assumptions. However, perhaps most critical 
to the delineation of information needs is the advance specification of 
those decisions for which data must be collected. 

Once the information needs have been delineated, a plan for 
obtaining the necessary data must be put into practice. Such a plan 
should include methods for collecting, organizing, and analyzing the data 
on both the local and the international levels. Certain facets of local 
preparat iori, aggregation, and reduction of data will have to be controlled 
if the problems related to the Association's analysis of the data are to 
be kept to a minimum... 



Finally, a plan for reporting the results must be developed. 
Such a plan should take into consideration the frequency, composition, 
and process of issuing local reports; the dissemination of local reports 
to other member organizations; public dissemination policies; the use 
of information for obtaining development funds; and an international 
timeline for process, interim product^ and terminal product dissemination. 



THE OBJECTIVE OF AN EVALUATION 

Before proceeding any further, it might be appropriate to 
consider the general objective cf such an evaluation — that is, what 
should the evaluation accomplish? Evaluation has already been defined, 
as has its relationship to the decision maker. ' What have yet to be 
examined, however, are the types of decisions for which evaluation data 
should be provided. Stufflebeam suggests that four broad types of decisions 
exist: planning, structuring, implementing, and recycling; so it might 
be stated that the general objective of an evaluation is to provide sufficient 
information to allow for the be%t possible decisions of each type to be 
made. 

The Association is past the embryonic stage; its shapers have 
labored long and hard in developing the blueprint for its existence. 
For this reason, it is contended that an evaluation, albeit informal, 
has been on-going; and further that this evaluation has provided useful 
data for making planning decisions, for the Association's objectives 
have been clearly stated. Still to be dealt with in formulating an 
evaluation strategy is the provision of information for structuring 
decisions, for the design of procedures; and recycling decisions, to 
react to or Judge the results. 

In providing for these types of decis ions , the evaluation 
should produce both the descriptive and judgmental data described by 
Stake in his ''Countenance Model." Though the use of judgmental data 
might be several years away, it will be a particularly crucial point in 
the life of the Association and should be planned for carefully. Recycling 
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decisions might be made as a result cf judgments based upon either 
relative or absolute comparisons, on the local or international levels. 
As such, the evaluation must provide not only data capable of documenting 
the procedures and describing their implementation, but data which will 
serve as a yardstick in determining the very viability of the concept 
of experimenting schools. 



4 
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AREAS TO BE EVALUATED, QUESTIONS TO BE ANSWERED 

About which areas must data be generated and what specific 
questions are to be answered? Though the response to these concerns will 
surely vary from member organization to member organization, the very 
reason for founding the Association requires it to develop a single 
definitive set of questions for use with all member organizations. 
However, until such a set of questions has been delineated, (and it should 
be considered a primary task), the proposal itself may be used to help focus 
upon areas of concern. 

For example, the Association's notion of the school as an^ 
experimental proposition depends upon six stated conditions being present 
in each organizational unit: 

1. A capacity to specify and agree upon the goals or 
crsterion selected 

2. A capacity to pose alternative ways in which the 
goals can be achieved 

3. A capacity to identify and agree upon means for 
Judging the degree of achievement of the goals 

A capacity to accept the possibility of failure 
of any alternative means tested 

5* A capacity to search for successful means of 
attaining the goals 

6, A capacity or attitude and expectation of 

adaptability to insure continuous institutional 
renewal • 

How one determines whether the above sir* capacities are present in each 
organizational unit has yet to be decided. The fact remains, however, 
that it will be necessary to provide data describing the conte.xt of each 
unit (member organization) from which one might easily ascertain the 
existence of those capacities. 
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Student output descriptors appear in the proposal in greater 
profusion and more succintly stated than any other type. From these 
descriptors, behavioral objectives might be generated, and questions 
related to those objectives, in turn, developed. For example, it is 
stated that "all persons who would live effectively in modernizing 
societies have certain capacities:" 

1. Openness to experience and adaptability 

2. Self-Identity 

3. Literacy and numerancy 

4. Ability to relate to others 

5. Ability to defin^ and solve problems 

6. Ability to contribute to the community 
7» Ability to learn readily 

8. Ability to use cognitive processes 
Since the idea of a person 1 iving effect Ively in a modernizing 
society is central to the mission of the Association, it will be 
necessary to evaluate students and student-related processes to 
determine when, why, and to what degree they gain those abilities. 

Other student output descriptors are mentioned for self- 
development and for coping. For self-development, each student needs 
the capacity (1) to be aware or to sense, (2) to understand and 
comprehend, and (3) to engage the environment. In order to cope^ an 
Individual is said to require (1) self-awareness, (2) values, (3) a 
repertoire of behaviors, (4) the capacity to predict, (5) self-discipline, 
and (6) the capacity for social Interaction. 

It is proposed that along with self-development and the coping 
capacity, comes a reduction in the Incidence of human failure. 

ER?C 



Specifically, individuals would be less subject (1) to the failure syndrome 
associated with schooling, (2) to mental illness, (3) to isolation and 
anomie, {h) to fatalism, and (5) to anit-social behavior. 

If data about these descriptors are truly important, they must 
be so stated and methods for collecting and measuring such information 
must be developed. 



THE GREAT INSTRUMENTATION COP-OUT 



Perhaps the most frequently used and often abused reason for 
not bothering to provide for the evaluation of alternative schools is 
related to instrumentation. Much of the blame for this, however, should 
be placed squarely on the shoulders of the educational research and 
evaluation community. The Insistence that evaluation equals measurement, 
and that unless a program can be measured with "hard" (direct) instruments. 
It is "no good,'* have caused many alternative school directors to avoid 
contact with so-called evaluators at all costs. In a position paper, 
Rosen has stated that "In ona recent case, an alternative school's goals 
were collected and then 'revised' to fit aval lable instrumentation, and it 
fs my impression that this is not an uncommon practice," 

There are a number of instrument-related reasons for not 
evaluating often cited, which, if examined closely, can be countered. 
For example, the emphasis on the use of behavioral objectives has led 
certain individuals to view evaluation only in terms of product assessment, 
and this, complains many project directors, falls to meet their needs 
because they are not merely interested in what happened, but also In how 
it occurred. However, total reliance upon evaluation of output is quite 
unnecessary, since most basic evaluation models provide for what 
Stuff lebeamcal Is process evaluation, or what Provus calls operation 
evaluation and interim product evaluation. This process of monitoring 
what is happening while it is happening and determining why it is 
happening, is a prime ingredient in the operation and management of 
any program. 

Another common complaint is that only highly trained specialists 
can evaluate alternative programs, and that such specialists are either 

ER?C .,5- 



not available or too expensive. Maybe highly trained evaluation 
specialists are needed to design a suitable evaluation, but even at the 
design stage they must work closely with program personnel. The most 
labor-intensive (and, therefore, most expensive) part of many evaluation 
designs is the process or monitoring stage(s). Often a majority of the 
responsibilities associated with this stage can be carried out by project 
personnel, including teachers and students. It is generally sufficient 
for an evaluator to supervise the collection of data, using available 
resources to actually complete the process. 

The unavailability of specific evaluation instruments should 
not be seen as a reason why avoid evaluating a program. Often", com- 
binations of existing instruments can be tailored to do the job. Just 
as often, however, upon checking the most recent literature, one may find 
mention made of useful new affective measures or culture-fair tests. 
Criterion referenced measures may also prove valuable. 

This raises the spectre of standardized or norm-referenced 
tests as the only "va? id*' measure of pupil growth. The fear of total 
reliance upon standardized tests has driven many alternative school 
directors to the opposite extreme--the belief that no standard i zed 
instrument can ever be of use — which is just as futile a position. 
Both '*hard*' and "soft'* data play a role in the evaluation of alternative 
programs. As Hickey has stated, "We must accept the fact that indirect 
measures become as important to evaluation and program improvement as 
conventional direct ones." One type of measure should not be "played-off" 
against the other, however. 
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NEXT STEPS 

Whether an International Association of Enper iment i ng Schools 
for Adolescents is to be established is no longer a question. At a 
meeting held in Philadelphia on September 15, 1973 it became a reality. 
Since that time, a half-dozen European nations have expressed an interest 
in joining the founding American, Canadian, and Israeli school organizations 
in their pursuit of solutions to a variety of basic social issues, ,Whdt 
remains to be determined, however, is whether the Association can serve 
as a catalyst for educational development in each of its mernber organizations. 
The design and i nstal lation^of a strong but flexible evaluation system is 
fundamental to monitoring the growth of each member organization, focusing 
and refocusing parts of each plan, and sharing succes^^;es and failures in 
this international endeavor. 

If the evaluation system is to realize its potential it must 

1, Be designed prior to the formal start of the experiment, 

2, Be acceptable to each member organization. 

3* Provide decision data useful at all organizational 
(structural) levels, from the individual to the 
Association, 

4. Provide decision data useful for the self-development 
of individuals at each personal (participant) level, 
including students, teachers, parents, administrators, 
and planners . 

5, Provide for the internal self improvement of each local 
organ izat ion , 

6, Provide decision data useful in comparing local experiences 
from the broader^ international frame of reference, 

7. Provide sufficient data both **hard*' and ''soft,^* to convince 
all of the program's '^publics" that it deserves continuing 
support* 
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8, Document the process of development of each member 
organization, 

9, Provide for the development and/or selection of 
appropriate measurement devices of instrumentation 
to be used during each phase of the evaluation, 
including the monitoring of the process 

10, Provide for the development of . a dissemination 
system for use at all levels. 

11, Be supported, philosophically and financially 
throughout the life of the experiment, 

12, Be subject to refocusing or revision as a result 

of periodic audits using such criteria as reliability, 
objectivity, internal and external validity, relevance, 
si gnif icance, scope, credibi 1 i ty, timel iness, and 
pervas i veness • < 

This paper has attempted to suggest some of the problems and 
examine some of the processes inherent in the development of an evaluation 
system for an International Association of Experimenting Schools. Much 
of what has been articulated here, however, will apply to any true 
alternative school or program. No matter what the scope or how many 
levels may be involved, the provision of information upon which to 
make decisions is central to the success of any such endeavor. 
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